Stochastic Variance Reduced Optimization for Nonconvex Sparse Learning

We propose a stochastic variance reduced optimization algorithm for solving a class of large-scale nonconvex optimization problems with cardinality constraints. Theoretically, we provide sufficient conditions under which the proposed algorithm enjoys strong linear convergence guarantees and optimal estimation accuracy in high dimensions. We further extend the analysis to its asynchronous variant to demonstrate a near linear speedup in sparse settings. Numerical experiments demonstrate the superiority of our method to other state-of-the-art results in both parameter estimation and computational performance.